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ABSTRACT 

This guide is designed to serve as a resource for 
vocational education administrators and educators involved in 
planning and evaluating Vocational programs on the state or local 
level. The first chapter deals with the history and benefits of 
cooperation between vocational education and the private sector. 
Addresse.d in the remaining chapters of the handbook are procedures 
for involving business ^ industry, and labor in vocational programs; 
creating initial involvement; planning together; evaluating together; 
and working together using techniques for effective involvemant. 
Appendixes to the guide include descriptions of the data collection 
procedures used during compilation of the manual and a series of 
case-site reports. Also included in the guide are se/en figures, 
three worksheets, a list of further readings, and a reference list 
(MN). 
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FOREWORD 



The Nation's leaders in both the executive and legislative branches of local. State, and Federal 
governments have shown unpreced'ented concern over the creation and retention of jobs for the 
civilian work force. This concern has been accompanied by a focus on educational excellence and 
the demand for vocational education programs of high quality. 

Government officials and educational leaders have underscored a major commitment to sup- 
. port economic development. High-quality vocational education and training programs may well be 
critical to this economic development thrust. Moreover, the involvement of business, industry, and 
labor in the planning and evaluation of these programs has been viewed as essential if this goal is 
to be fully realized. 

Public secondary and postsecondary vocational education programs are responding by initiat- 
ing and improving programs that support business, industry, and lab-ii" requirements for upgrad- 
ing, retraining, and job training in new skill areas. Close cooperation among educators, business, 
industry, and labor in specific planning and evaluation activities remains a high priority. 

This publication is a resource for vocational education administrators and educators, and 
others involved in planning and evaluating vocational programs on the state or local level. It con- 
tains ideas and suggestions for initiating or improving the involvement of business, industry, and 
labor in vocational education planning and evaluation. It also provides strategies and techniqi^es 
for use in promoting this involvement. 

The National Center for Research in Vocational Education, The Ohio State University, under 
contract with the Office of Vocational and Adult Education, U.S. Department of Education, con- 
ducted this project. It was carried out in the Evaluation and Policy Division of the National Center 
under the direction of N. L. McCaslin, Associate Director. 

The National Center is particularly indebted to Stephen J. Franchak, Project Director: J<ianne 
Desy, Program Associate: and E. Lee Norton, Graduate Research Associate, who prepared this 
document. Significant contributions to the development of the final manuscript were also made by 
N. L. McCaslin, Associate Director: Ann Nunez, Research Specialist: and Morgan V. Lewis, Senior 
Research Specialist. Appreciation is extended to Pat Fornash, Graduate Research Associate, for 
her valuable contributions in the development of information and the conduct of project tasks 
relating to the development of the document. 

Recognition is also given to Ida Halasz, Research Specialist; Bill Stevenson, Senior Research 
Specialist; Catharine Warmbrod, Research Specialist; Jill Frymier Russell, Program Associate: and 
Karen Behm, Graduate Research Associate, for their collection and analysis of data and informa- 
tion in the conduct of this project. 

Appreciation is extended to those at the National Center who reviewed the first draft of this 
document, Michael Crowe, Research Specialist, and Harry Drier, Associate Director. Donald M. 
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Clark, President, Nath mal Association for Industry-Education Cooperation, and Merrill L. Meehan. 
Educational Research and Development Specialist, Appalachia Educational Laboratory, also 
reviewed the draft copy. 

A special note of thanks is extended to Mary Beth Dauner for her preparation of the final 
manuscript and to Marjorie Arnold, Priscilla F. Ciulla, Deborah Anthony. Sherri Trayser, Katherine 
Haycook, and Marilyn Orlando for their secretarial assistance. Editorial assistance was provided by 
Janet Kiplinger and Judy Balogh of the Editorial Services staff. 



Robert E. Taylor 
Executive Director 
The National Center for Research 
in Vocational Education 



viii 



8 



EXECUTIVE SUMMARY 



Cooperation among business, Industry, labor, and education has become a key concern of the 
1980s, a time when scarce resources have led many diverse groups to work together toward com- 
mon goals. Studies and samplings of popular opinion indicate that Americans believe public voca- 
tional education should increase and improve its involvement with the private sector, particularly 
employers and labor representatives, in order to facilitate the flow of workers from educational 
programs to jobs. 

Public vocational education and business, industry, and labor have worked together in the 
past and continue to do so, but this involvement has too often been fragmented and uncoordi- 
nated. Joint planning and evaluation, a primary goal of both education and the private sector, sel- 
dom takes place in any formal manner. 

This publication provides specific guidelines for carrying out such joint planning and evalua^ 
tion. The intent throughout is to encourage local and state educational personnel to increase their 
coordination and collaboration with business, industry, and labor for the purpose of improving 
vocational education programs. 

The procedures, strategies, and techniques given here are based on analysis of existing data 
augmented by three comprehensive case studies. A second analysis of existing data was con- 
ducted to examine the perceptions and conditions surrounding private sector involvement in voca- 
tional education planning and evaluation. Finally, additional general information was obtained 
through a review of selected literature. These case studies involved personnel from both public 
vocational education and the private sector in discussions that focused on the mutual involvement 
of these sectors. 

Because of the wide variety of information and suggestions included in this handbook, it is not 
possible to Include all the significant points here. Instead, a number of points of special impor- 
tance are given in the remainder of this summary. 

The primary mechanism for public vocational education and private sector involvement in 
planning and evaluation has been the collaborative council or advisory committees. A council or 
committee should include as wide a variety of viewpoints and interests as possible Ideally, repre- 
sentatives should be future-oriented people who are recognized and respected by their peers. 
Every major facet of the local business, industry, and labor communities should be represented. 
Employers and employees should be represented both by people who have years of experience 
and by people who are new entrants into their occupational fields. 

When a group convenes, an orientation meeting should take place. School administrators 
Should familiarize representatives of the private sector with the overall educational program and all 
relevant details of the vocational program. It is important to discuss at the outset the nationally 
recognized goals of education and how the local and State programs address these goals. 
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Representatives from business, industry, and labor should also clearly define their roles as 
they see them and their overall goals. All representatives should become familiar with each other's 
systems, organizational structures, governances, and finances. The ultimate objective of this orien- 
tation is to begin building unity and involvement through describing how vocational education fits 
into the overall industrial environment. 

After this initial exchange of information, the work of the group may very well center on a 
thorough examination of the vocational education system with attention given to the following 
activities: 

• Examination of instructional content 

• Assessment of instructional activities or methods 

• Review of facilities, equipment, and materials for instruction 

• Examination of program costs and expenditures 

• Review of opportunities for inservice and preservice training of teaching staff 

• Identification of supplemental education needs of employed workers in the community 

• Examination of ancillary or support services, vocationally oriented guidance (i.e., career 
awareness and exploration), and job placement services 

Representatives of business, industry, and labor who agree to become involved in vocational 
education planning and evaluation expect to be asked to deal with substantive issues The group's 
work should be purposeful and well defined. Representatives should also expect to provide sug- 
gestions and participate in determining the scope of the group's activity rather than have agendas 
and goals continually defined for them. 

Five major strategies that help establish close collaboration of all involved in planning and 
evaluation are as follows: 

• Increase communication. 

\ • Provide a mechanism for joint planning and decision making. 

] 

I • Ensure commitment from involved agencies or persons, 

o Define the benefits for all agencies or persons involved. 

• Establish legal mandates. 

These strategies provide a framework for involving business, industry, and labor in vocational 
education in order to improve programs and satisfy the needs of the labor market. The collabora- 
tion that can result from the use of these strategies can provide valuable assistance to educational 
institutions in numerous specific ways. It can, for example, help vocational education— 

• determine appropriate instructional objectives; 

• identify improved curriculum and instructional methods, 
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• motivate students and teachers; and 

• develop channels for procuring and updating facilities, equipment, and supplies. 

Involvement can be equally advantageous to business, industry, and labor, since it can 
enhance the potential of vocational education to increase productivity, develop a satisfactory work 
ethic in participants, and increase job satisfaction. 

For all concerned, successful cooperation and collaboration among business, industry, labor, 
and education offer many advantages. Ultimately, planning and evaluating the vocational "'ogram 
together can lead to very important benefits for the community. Improved vocational ea-. lon 
can create a supply of better-trained and more-productive workers to meet the needs of business 
and industry and enhance economic development. 
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PURPOSE OF THE HANDBOOK 



This handbook is designed to encourage local and State educational personnel in their efforts 
to coordinate and collaborate with business, industry, and labor for the purpose of program 
improveTient. Toward this end. the handbook identifies successful strategies and procedures used 
by secondary and postsecondary vocational educators to initiate and increase the involvement of 
the private sector. Finally, it provides specific guidelines and procedures for carrying out the joint 
planning and evaluating of vocational programs. 

tn developing the handbook, four major questions were stressed: 

• What changes are needed in the planning and evaluating of secondary and postsecondary 
vocational education if input from business, industry, and labor is to be included? 

• What specific decision-making activities should vocational educators arid representatives 
of business, industry, and labor undertake in planning and evaluating secondary and 
postsecondary vocational education? 

• How can vocational education, business, industry, and labor implement planning and 
evaluating activities for secondary and postsecondary vocational education? 

• What process activities address the involvement of vocational educators with representa- 
tives of business, industry, and. labor in evaluating secondary and postsecondary voca- 
tional education? 

Many individuals and groups provided input on these issues. The analysis of existing data 
from business, industry, labor, and education by various local. State, and Federal agencies pro- 
vided the initial information base. The National Certter's previous work in business, industry, and 
labo' coordination and collaboration also supported the development effort. Additionally, informa- 
tion obtained from the three case site visits provided ideas to support the development of the 
strategies, procedures, and techniques. Finally, telephone discussions and mail correspondence 
with 34 State and local vocational education personnel provided input on relevant examples of 
business, industry, and labor involvement in vocational education planning and evaluation. 

The strategies and techniques given here were developed from a content analysis of the 
above-mentioned information sources. That information base was refined and validated through 
the nominal group process with 14 persons from the National Center. These persons were expe- 
rienced in vocational education evaluation and planning as well as in collaboration with business, 
industry, and labor. Finally, a panel of four persons representing business, industry, labor, and 
education at the local and State levels was asked to critique the proposed strategies and tech- 
niques. From this exchange of information a final list was developed. 

This handbook describes private sector involvement in educational planning and evaluation 
from several perspectives, beginning in chapter 1 with an overview of the history of this involve- 
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ment and the trends affecting it today. Chapter 2 provides a description of industry-education 
work councils and vocational education advisory committees. Both barriers and facilitators to 
Involvement are also identified in this chapter. Methods for creating initial involvement are given in 
chapter 3. Chapters 4 and 5 discuss steps in planning and evaluation, with attention given to legis- 
lation that affects vocational education planning and evaluation. Chapter 6 presents specific tech- 
niques for working together in councils or advisory committees. These perspectives provide a 
guide that vocational education personnel can use in working to involve business, industry, and 
tabor in effective program planning and evaluation. 



CHAPTER 1 



VOCATIONAL EDUCATION AND THE PRIVATE SECTOR 



Most people agree that vocational education has contributed significantly to the employment 
and training of American youth. This belief was indicated by responses to the Gallup Poll of Public 
Opinion on Public Education (Gallup 1983). Overall, those surveyed expressed a high degree of 
support for the idtea of making vocational education available to all youth. 

Research indicates that this confidence is justified. Students who have enrolled in vocational 
education programs experience greater rates of participation in the labor force, less unemploy- 
ment, and higher annual incomes (Gardner, Campbell, and Seitz 1982). For graduates from most 
program areas, the income advantage is long term and large enough to be of practical significance 
(Desy. Mertens. and Gardner 1984). , 

Much of the effectiveness of vocational programs can be credited to the responsiveness voca- 
tional educators have shown to changing labor market demands. The number of vocational pro- 
grams directed toward skill shortages in specific occupations has grown over the past several 
years (Starr 1983). Vocational educators have developed numerous approaches toward facilitating 
the school-to-work transition, enhancing productivity, and contributing to job creation. The pro- 
grams that have emerged include cooperative education, work-study, customized job training, 
quick-start programs, industry-education, work-education, and area planning councils. Many are 
designed to include business, fndustry, and labor at every possible level. 



The History of involvement 

Private sector involvement in vocational programs is not new. Vocational education has been 
linked with business, industry, and labor since the Smith-Hughes Act of 1917. The interdepen- 
dence of business, industry, labor, and education was formally recognized in the rules and regula- 
tions issued by the U.S. Office of Education in 1922 (Burt 1967). These linkages were so close that 
early vocational education programs were sometimes criticizl9d by those who thought job training 
closely aligned with business needs was not always in the best interests of the people. 

The role of business, industry, and labor in education has changed with changing social 
needs. In the 1960s and early 1970s, the Federal Government emphasized job training as a means 
of affecting social mobility rather than a vehicle for overall economic growth. When the private 
sector was involved with education, it likewise tended tb address issues of social equality. Advisory 
groups often reflected the concern for democratic representation of underserved groups. The 
focus had shifted from the private sector interests of earlier times to the social issues of the day. 

Recent years have seen another shift in both the nature of vocational education and the 
demand for private sector involvement. Demographic trends and advances in technology have 
created an increasingly urgent need for the involvement of business, industry, and labor in voca- 
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tional education planning and evaluation. Clark and Rinehart (1982) note this increased Interest, 
but add that the joint efforts of the private sector and vocational education are often fragmented 
and uncoordinated. These efforts, they conclude, must be unified if the linkage between education 
and work is to become stronger. 

The possibility exists that the link will become weaker instead. Because some educational and 
training needs have not been met by the educational system. United States employers in the pri- 
vate sector are now spending between $30 and $40 billion a year on human resource development 
and training programs (Craig and Evers 1980). The involvement of vocational education has the 
potential for enhancing the cost-effectiveness of these human resource development and training 
programs. 



The Influence of Social Change 

Shifts in the interface between education and work are the result of major social change. The 
most striking changes in the United States are those resulting from the long-term decline In fertil- 
ity (Coates 1983). This demographic trend has led to smaller families, a higher percentage of older 
persons in the population, and fewer young people entering the work force. The number of Ameri- 
cans between 16 and 24 years old peaked at about 36 million in 1983. The U.S. Department of 
Labor expects that figure to fall by 10 percent through 1985 and another 7 percent by 1990. One 
effect of this trend is that the focus of skilled work force preparation has increasingly shifted to the 
retraining of adults, including women and older citizens. 

A second major influence on the interface between education and work is technological 
change. Because of new technology, job skills in many occupations are becoming obsolete at a 
faster rate than ever before. Automated equipment is displacing large numbers of lower-skilled 
workers and some more highly skilled workers as well. The same advances in technology have 
sharply increased the demand for higher-level training and retraining programs. Many more jobs 
now demand a comprehensive grounding in mathematics and the sciences. 

Worthington (1981) cites additional major changes that will continue to affect the relationship 
of education and work. The first is the fight to halt inflation and revitalize the economy. The 
second is a general reliance on the private sector— primarily on employers— for training. These 
developments, he concludes, will result in the following changes: 

• Elementary and middle schools will strengthen their emphasis on career development 
education. — — 

• Educational institutions will reemphasize the importance of mathematics and science. 

• Postsecondary institutions will place more emphasis on technical education and receive 
new forms of support from businesses and industries, especially the use of expensive 
equipment. 

• Private employers will increase their own training efforts, but will also support public 
vocational education, recognizing the need for this source of entry-level skilled workers. 



Tht Btntflti of Involvemtnt 



In this climate of change, closer links between business, industry, and labor offer many advan- 
tages. One is better communication of information about local job and training opportunities 
(Prager et al. 1981). As Clark and Rinehart (1982) point out, such communication benefits all par- 
ticipants. For industry these benefits include a supply of potential employees trained to meet labor 
market needs. Industry can also design and implement cost-effective training programs for current 
employees with the help of vocational educators. Vocational students benefit from better linkage 
through up-to-date training, work experience that relates theory to practice, and potential job 
opportunities. Their Instructors benefit by receiving state-of-the-art information and resource sup- 
port from the private sector. Educational institutions benefit through better information to use in 
updating instructional programs. Finally, vocational educators are better able to improve their 
training and placement programs. 

In the long term, society as a whole benefits from private sector involvement in vocational 
education. The ideal linkage balances the interests of individuals, business, industry, labor, and 
the community. 



5 



16 



CHAPTER 2 



INVOLVING BUSINESS, INDUSTRY. AND LABOR 
IN VOCATIONAL PROGRAMS 



Typts of InvolvtmenI 

Relationships between vocational education and the private sector have been the subject of 
careful study. The literature in this area uses various terms to describe cooperation, with coordina- 
tion, linkage, and collaboration among the most common. 

• Coordinallon— This process begins with those agencies involved identifying their com- 
mon goals for maintaining organizational effectiveness. Once they have done so, they 
interrelate services toward these common goals, without sacrificing individual program 
goal» or requirements (Maurice 1981). 

• Linkiige-'A negotiated, authoritative arrangement, linkage takes plac^ between organiza- 
tions able to coordinate and/or exchange resources or activities. The purpose of linkage 
is to achieve each organization's goals and also meet mutually defined goals and objec- 
tives (Banathy and Duwe 1978). 

• Collaiborallon— This process depends on all parties involved clearly defining their expec- 
tations. When they have done so, the next step is reaching agreement on shared goals. 
Those goals define the group's actions from that point on (Hord 1983). 

These three terms are often used in defining ways business, industry, labor, and education can 
work together. The process oj networking makes use of all three kinds of relationship, as well as a 
fourth, cooperation (Colorado Department of Education 1983). Networking further involves con- 
nections between people who can share information and resources with one another. Networks 
may well be the structures of the future (Naisbitt 1982). At the least, their usefulness is undeniable. 
Any attempts to involve the private sector with vocational education should make use of network- 
ing as far as possible. 



Cooperative Practices 

The most common forms of cooperative practice between the private sector and vocational 
education are advisory committees and collaborative councils. Advisory committees are usually 
mandated through State and Federal legislation. Collaborative councils, in contrast, are voluntary 
organizations. 



Advisory committMS. Rogers (1981) describes an educational advisory committee as follows: 

An organized group of lay people interested in education who give assistance in plan- 
ning an educational program that will prepare students for satisfactory employment in 
the business community, (p. 1 1 ) 

The literature on advisory committees often stresses the idea that a consistent exchange of infor- 
mation is necessary to keep educational programs relevant and accountable. Advisory committees 
are a good vehicle for this interchange. A number of the strategies and techniques for involving the 
private sector in vocational education given in this handbook can be used in advisory committees, 
and may sdrve to fill the need for information in this area. 

Advisory committees can serve a very wide variety of functions, According to Scheller (1978) 
these can iriclude— 

• public relations, 

• curriculum advice, 

• funding support of the program, 

• support of legislation, 

• program evaluation, 

• recommendations for change, 

• textbook reviews, 

• encouragement of individual teachers, and 

o 

• placemefnt selection criteria. 

The functions an advisory committee performs will vary with the people involved and the spe- 
cific needs of the vocational program. Hagman and Barlow (1971) specify four-broad functions 
advisory committees can perform. The first three— public relations, instructional planning tasks, 
and program evaluation— are included in the above list; Hagman and Barlow add job market analy- 
sis as a fourth important function of advisory committees. Other sources, including the Pennsyl- 
vania Advisory Council on Vocational Education (1982) and Riendeu (1977), have str&ssed that a 
basic function of the advisory committee should be to give advice in the development of both long- 
range and annual plans, an approach that is also emphasized in these guidelines, 

The effectiveness of advisory committees has often been questioned. According to Oswald 
(1983), a statewide study of 1,000 vocational educ ation advisory committees indicates that these 
committees have little or no impact on the vocational programs they serve. Oswald adds that this is 
a critical problem. Job obsolescence, shifts in the work-age population, and changes in the econ- 
omy make it more necessary than ever to link vocational education to the community through 
advisory committees. 

Collaborative councils. A second major form of cooperative practice is the community-based 
council (Clark 1982). Industry-education councils and work-education councils typically play far 
more active roles than advisory committees. Clark describes the role of these councils: 

8 
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The linkage between education and the broader community through an industry- 
education council highlights the direct participation of decision makers from the private 
and public sectors working with the schools In such key areas as staff and curriculum 
development, occupational Information systems, school-based job placement, and work 
experience/career exploration. This industry-education alliance focuses on thorough 
planning, realistic objectives, key result areas, incremental improvement, and 
measuremenrs—supported by a strong long-term commitment by education and the 
business/iabor/government/professional community, (p. 25) 

Despite this potential, not every council uses its energies to the best advantage. Countless 
hours may be spent in a council "selling" the concept of collaboration, building constituencies, 
talking through problems, building consensus, and sustaining enthi|siasm (Elsman 1981). 
Moreover, like advisory committees, councils are not always effective. Among the problems some- 
times encountered in operating councils are institutional inertia and suspicions between the 
groups involved (Useem 1981). 

This should in no way discourage educators or members of the private sector from forming 
councils. There are usually some difficulties in any form of collaboration, and a council mpy still 
provide the most appropriate context in which business, industry, labor, and education representa- 
tives can work together (Clark and Rinehart 1982). 



Barriers to Involvement 

Forming a council* can be difficult unless representatives of the private sector feel some 
degree of responsibility for solving community educational problems. If they do not, it may be 
Impossible to build enthusiasm for cooperative endeavors. Without a sense of responsibility, there 
is little incentive for participation in an advisory committee or collaborative council, it is vital, tht?n, 
that those who are initially chosen to form a council share a commitment to the vocational educa- 
tion process. 

When it is formed, a council Is a collection of people from diverse organizations with e-'jually 
diverse interests. This fragmentation must gradually be transformed to a sense of unity, if effective 
working relationships and involvement are to develop (Clark 1983). The council can become uni- 
fied if representatives are given a coherent structure and a process leading to involvement. One 
good approach is to direct the council toward specific activities, such as school improvement or 
economic development, that benefit employees, youth, and adults, and that can fit into the inter- 
ests of most or all the members. 

Members from the private sector may have a different sense of the decision-making process 
than those from education. If the council moves too slowly, representatives from business, indus- 
try, and labor can become impatient. Key decision makers in the private sector usually recognize 
the need for thorough planning (Clark 1983); however, time is money in business, industry, and 
labor, and time spent in meetings without reaching any decisions may soon discourage private 
sector representatives and lead them to withdraw from the council. Any political controversy can 
also cause members to disassociate themselves from an advisory committee or council (Prager et 
al. 1981). 



•The word "council" will be used from this point on to refer to a cooperative group of representatives from the private sec- 
tor and education, regardless of specific structure. 
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Representatives from management and unions are sometimes reluctant to work together on 
the same advisory committee or council— possibly because of the adversarial relationships 
involved in the collective bargaining process, or because the employer hires nonunion labor. In 
addition, labor union representatives are often reluctant to become involved because they feel they 
cannot speak for labor as a whole. Forming a council demands tact and sensitivity to these issues. 

A problem often encountered by councils attempting to plan vocational programs is the lack 
of timely, accurate information from employers. New councils should be sure to address them- 
selves to the need for structured communications with business and industry (Clark 1982). A simi- 
lar consideration is the need to tap into the power structure of the community. It is especially 
Important to work with those employers who control the economic development process. A coun- 
cil's effectiveness can be greatly diminished if it does not have the level of management support 
within business that is needed to gain the attention of employers. 

This listing of the problems councils sometimes encounter is meant to be helpful rather than 
discouraging. The research mentioned here suggests that these barriers should be taken into 
account. Successful councils and committees have had to overcome these problems in order to be 
effective Strategies and techniques that can be used to form a council, overcome difficulties, and 
build effectiveness in planning and evaluation are delineated in the remainder of these guidelines. 
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CHAPTER 3 



CREATING INITIAL INVOLVEMENT 



The best way to create a unified, purposeful council Is usually to Involve the group in a mean- 
ingful task. Working toward a common goal tends to overcome diversity and minimize any difficul- 
ties members of various groups might experience in working together. Unity and (5ommitment are. 
best ceated through jsarticipation in the overall process of planning the vocational program and 
evaluating its success and its needs. 

In each step of this process, business, industry, and labor representatives should be involved 
to varying degrees with the level of involvement dictated by the role vocational education plays in 
the group the member represents. Members of the phvate sector are impacted ^y vocational edu- 
cation in different ways. It Is therefore essentiaJ that the council, In its formativ^ stages, examine 
the roles and functions of the individuals or firms who may be asked to represe^nt the private sec- 
tor. Those who are more directly affected by vocational education are likely to/have more interest 
in working In a council. / 



The Planning and Evaluation ^rocett 

Figure 1 gives an overview of the planning and evaluation process. The framework is con- 
structed to indicate that planning leads directly to evaluation. It further illustrates that at any point, 
from the initial description of needs to the assessment of the procedure, representatives. of busi- 
ness, industry, and labor can participate. 

Areas of participation should depend on expertise and interest. For instance, a labor represen- 
tative may have special concerns about the mission of vocational education, a business person 
may be knowledgeable about routes for Interagency coordination, and a member from industry 
may be interested in program needs. Of course, all members should be fully informed about the 
work of other members, and all should have ample opportunity for input in every area. 

There are numerous ways to structure the planning process. All emphasize certain fundamen- 
tal steps: assessing the context and mission, setting goals. Implementing strategies, and evaluating 
results. Evaluation includes reassessing the goals, assessing the outcomes, and reevaluating the 
overall plan. The steps within tKe planning and evaluation process should be as well defined as 
possible. Following is a more detailed breakdown of the steps which ars normally part of a careful 
planning and evaluation process. 

1. Steps In Planning: 

• Explore possible futures and examine values. 

• Establish goals (long-range). 
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Figure 1. A structural framework for involving butineis, 
Industry, and labor In planning and evaluation. 



SOURCE: Adapted from Starr, Merz, and Ilannlser (1982, p. 13). 
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• Sense opportunities and assess needs. 

• Determine capacities and constraints. 

• Specify objectives (short-range). 

• Set priorities under given constraints. 

• Develop alternative courses of action. 

• Analyze the alternatives. 

• Select a course of action. 

• Identify resource requirements. 

• Prepare an implementation plan. 

• Formulate evaluation/feedback mechanism. 

As this list shows, a complete plan includes a mechanism for evaluation. The initial exploration 
is also a form of evaluation, and must be reasonably comprehensive if the group is to address 
needs realistically and set appropriate goals. 

Like planning, evaluation follows a logical pattern. 

2. Steps in Evaluation: 

• Identify general focus of the evaluation. 

• Develop evaluation questions. 

• Identify evaluation criteria. 

• Specify evaluation methods. 

• Develop data analysis plan. 

• Collect evaluation data. 

• Analyze and interpret the data. 

• Prepare report on the results. 

• Develop dissemination plan. 

• Develop plan for assessing the evaluation. 
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Idtntifying Priority Ar«at 



Worksheet 1 presents a method for identifying the areas in which the vocational education 
system can most benefit from worl(ing with representatives of the private sector. One page is given 
to planning and one to evaluation, with a*l the tasks just presented included. The worksheet should 
be filled out by the vocational program administrator and other persons knowledgeable about the 
program and the context in which it operates. 

To use the worksheet, examine each task and rate it on the left for its importance to present 
needs. Once priorities are established for every task on the list, return to the beginning. This time, 
decide to what degree members of the private sector should be involved in each task. 

The completed worksheet will serve as a visible guide to priorities in involving business, indus- 
try, and labor in the vocational program. It should suggest people and organizations to contact for 
a new council. When the planning task is high priority and involvement by business, industry, and 
labor is also high priority, members should be very carefully selected, since their contributions can 
be quite important. 



) Strategies for Inyoivement 

Once the need and the priorities for involving the private sector in voc aticnal education are 
established, specific strategies to bring about this involvement should be v. n^^en. This section 
presents a set of strategies vocational education administrators have used either to initiate or to 
improve the involvement of business, industry, and labor in program planning and evaluatipn, 

Strategy is used here in the sense of a broad course of action. Strategies are chosen accord- 
ing to how well they address defined goals or problems. Techniques— particular methods of activi- 
ties used to support strategies— are discussed in detail in chapter 6. One or several techniques 
may be used to implement any given strategy. 

The best way to choose a strategy to involve others in vocational education is to examine the 
specific planning and evaluation tasks selected on Worksheet 1 as having high priority. The plan- 
ning task and the degree of involvement desired will suggest the choice of strategies. It can be 
very helpful to discuss possible strategies with members of the private sector and others in educa- 
tion. After consideration, a specific strategy for long-range involvement should be chosen. 

The strategies presented here focus on what seem to be the fundamental issues of building a 
cohesive group. They are— 

• increase communications, 

• provide mechanisms for joint decision making, 

• build commitment in involved participants, 

• define the benefits of participation to those involved, and 

• establish legal mandates. 
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WORKSHEET 1 

ESTABLISHING PRIORITIES FOR INVOLVING BUSINESS, INDUSTRY, 
AND LABOR IN PLANNING AND EVALUATION 



Importance 
of Activity 



Degree to Which 
Business, Industry, 
and Labor Should 
Be Involved 



High 



Low 



5 4 3 2 
(check one) 



PLANNING TASKS 



High 



Low 



5 4 3 2 1 
(check one) 



• Explore possible futures and 

examine values. 

• Establish goals (long-range). 

• Sense opportunities and assess 

needs, 

• Determine capacities and 

constraints* 

• Specify objectives 

(short- r-ange)* 

• Set priorities under given 

constraints. . 

• Develop alternative courses of 

action. 

• Analyze alternatives. 

• Select course of action. 

• Identify resource requirements. 

• Prepare an implementation plan. 

• Formulate evaluation/feedback 

mechanism. 
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WORKSHEET 1 -Continued 



Importance 
of Acti\^ity 



Degree to Which 
Business, Industry, 
and Labor Should 
Be Involved 



High 



Low 



5 4 3 2 1 
(check one) 



EVALUATION TASKS 



High 



Lo\N 



5 4 3 2 1 
(check one) 



Identify general focus of th(i 
evaluation. 

Develop evaluation questions. 

Identify evaluation criteria. 

Identify evaluation methods. 

Develop data analysis plan. 

Colleci evaluation data. 

Analyze and interpret data. 

Prepare report on the results. 

Develop dissemination plan. 

Develop plan for assessing 
the evaluation. 
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The first and most basic strategy for promoting involvement in a group effort is to increase 
communications. This can be done in many ways. A survey can be used to determine public opin- 
ion on the type and degree of involvement that seem desirable. An information exchange network 
can be established to gather the data needed for planning and evaluation. The community can give 
input through regular open hearings focused on planning and evaluation issues. Another way to 
increase communication^ m'lih the community is to generate instructional information for teacher 
education institutions, apprenticeship programs, or business, industry; and labor Associations. 

A second strategy for increasing involvement is to provide mechanisms for joint decision mak- 
ing. One good beginning is to set up a committee to identify and select goals for the vocational 
programs. Another possibility is to conduct group sessions on selected topics for representatives 
of businesf, industry, labor, and education, with the sessions ending in consensus or resolution. 
St^ll another way to initiate ioint decision r.aking is to set up specific cullaborative. decision- 
making activities. These can involve such programs as cooperative education and industry 
exchange. <• * 

A third importer* strategy is that of building commitment among those who are. or could be, 
involved. This can oe fostered by obtaining the endorsements of the recognized authorities or 
"power brokers" who command respect within the business, industry and/or labor constituencies. 
Commitment can also be built by providing special recognition for those who contribute to plan- 
ning and evaluation efforts. 

A fourth strategy for involving members of the private sector is that of defining the benefits of 
participation. This means appraising and discussing the potential benefits of their planning and 
evaluation tasks with the participants. Cost-benefit estimates are usually important to the private 
s fl r .tor , and sh ould be includedJttJie]:e_ap|u:Qpriate. The benefits actually derlved-byspecifie^aftic- 
ipants can be publicized. 

A final strategy is the establishing of legal mandates. This means formulating policy state- 
ments addressed to State legislatures, labor unions, corporations, and State and local school 
boards. Such activity publicizes the needs of the vocational program as well as increasing the 
commitment of those who speak out for the program. . 

Strategies must be flexible and open to change. Over time, the need for involvement varies and 
tVe availability of resources changes. The selection of strategies for involving community sectors 
i hould be responsive to these changing needs. 

Figure 2 presents an overview of the process of identifying first tasks, then strategies, then 
techniques. The techniques listed are described in chapter 6. Note that the selection of tasks leads 
to the involvement of those representatives of business, industry, and labor who can contribute to 
the choice of strategies and techniques. 



Initiating Involvement 

Advisory committees and councils are usually formed by one or more key people from busi- 
ness, industry, labor, O" education. For the council to be effective, however, initiative must be 
taken over by members from all the groups involved (Iowa Career Education Division 1983). 

Initial momentum is important. This is sometimes established by a small subcommittee, with 
school board members playing a vital role. Such a group can develop statements of purpose and 
objectives that define the group and help focus goals. Choice of leadership during this period is 



17 



27 



Vocational Education 
PLANNING 
TASKS 



7^ 
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Vocational Education 
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TASKS 




STRATEGIES 



Increase communications 
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TECHNIQUES 



Audit 
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DACUM 
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Discussion 
Illustrated talk 
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Process cause and effect analysis 
Quality circle 
Survey 



Figure 3. A procnit framework tor Identifying strategies 
and techniques for planning and evaluation. 
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also very important. The overall effort Is best coordinated by an industry-education coordinator or 
director of school-industry cooperation— someone with experience in both industry and education 
(National Association for Industry-Education Cooperation 1973), Those who represent business, 
industry, and labor should have firsthand experience with the vocational programs they are advis- 
ing. Employers and employees should be represented by persons with years of experience in their 
fields as well as by new entrants. 

The council should include people with a wide variety of viewpoints and interests. Ideally, 
representatives should be future oriented. The more representatives are recognized and respected 
by their peers, the more easily the council can accomplish its work. Every major facet of the busi- 
ness, industry, and labor communities should be represented. 

When a group first convenes, an orientation meeting should take place. School administrators 
should familiarize representatives of the private sector with the overall educational program and 
present all relevant details of the vocational program. At this point it is important to discuss the 
nationally recognized goals of education and the ways local and State programs address these 
goals. 0 

In the initial meeting, representatives from business, industry, labor, and education should 
clearly define their roles as they see them and their overall goals. They should also begin becom- 
ing familiar with each others' systems, organizational structures, governances, and finances. The 
final activity of this initial orientation should be an exploration of how vocational education fits into 
the overall industrial environment. 

In its first meetings, the group will need to devote time to becoming thoroughly familiar with 
the vocational education system. Following are some areas of study: 

• Examination of instructional context 

• Assessment of instructional activities or methods 

• Review of facilities, equipment, and materials for instruction 

• Examination of program costs and expenditures 

• Review of opportunities for inservice and preservice training of teaching staff 

• Identification of supplemental education needs of employed workers 

• Examination of ancillary or support services, vocationally oriented guidance (i.e., career 
awareness and exploration), and job placement services 

In forming a council it is important to bear in mind that representatives of business, industry, 
and labor who agree to become involved in vocational education planning and evaluation expect 
to be asked to deal with substantive issues. The group's work should be purposeful and well- 
defined from the beginning. At the same time, representatives also expect to provide suggestions 
and participate in determining the scope of the work, rather than have agendas and goals continu- 
ally defined for them. 
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Th0 Efftetlv« Council 



Studies have been done to determine what makes councils effective, Datteri (1977) compared 
nine State councils that were considered outstanding with nine other State councils chosen at 
random to determine the differences betwden the two groups, and made the following recommen- 
dations as a result: 

• Councils consider meeting regularly at least 10 times a year. (This idea is supported by 
another study, which found that increasing the number of meetings per year increased 
the perceived level of importance given to the council by some members, particularly 
those from area vocational schools and community or junior colleges (Behymer 1977].) 

• Councils conduct at least four public heafings per year at various sites in the geographic 
area. Public hearings give the council valuable feedback and allow the community to par- 
ticipate in the decision-making process. 

• Councils consider employing at least two professional staff persons. One function of the 
staff should be to maintain excellent relationships with legislators and with State voca- 
tional directors and their staff. 

• Councils develop good working relationships with State education agencies and other 
related groups. 

• Councils build sources of information and input. 

Numerous factors affect the success of advisory committees and councils. Clark and Rinehart 
(1982) identified three factors that may be particularly important in the early phases of the council. 
First is the identification of those who will form the council. Second is clarity of expectations; 
those involved should understand both the services to be contributed and the length of the 
appointments. Last is consensus on the real purpose of the committee, a necessity if the group is 
to move toward common goals. 
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CHAPTER 4 



PLANNING TOGETHER 



There are few notable examples of comprehensive planning success. House, Shull, and 
Williams (1978) explain that this is so because planning rests on several factors not always within a 
< group's control: 

The U.S. Lunar Exploration Program is a shining example of good planning—NASA 
planning techniques were held up as models to all other federc. agencies for several 
years. However, NASA had a few advantages that most other agencies lack: (1) a con- 
cise goal (a man on the moon by 1970), (2) almost unanimous public support, (3) almost 
no budget constraints, and (4) little need for interagency coordination, (p. 4) 

Vocational education groups, whether they are local. State, or Federal, do not have these 
advantages. Their planning must address multiple goals and client groups, varied public support, 
restrictive budgets, and the need for interaction with both internal and external agencies. For these 
reasons it is important to carefully consider a number of critical questions and factors as a council 
forms and begins planning. 



Initial Contideratlont 

Long-range strategic planning is both complex and time consuming. Answering the initial crit- 
ical questions requires timely, detailed, and valid labor market information, as well as a variety of 
resources. Few vocational education administrators enjoy the luxury of a full-time or even a half- 
time planner to address these questions. Nevertheless, long-range strategic planning is necessary 
In order to satisfy program needs and provide for program improvement. 

Before inv ''Iving the private sector In the long-range planning process, an administrator 
should have a fit r\ grasp of the problems and Issues that are fundamental to the process. These 
critical questions and the information used to answer them can be shared with the convening 
council. The questions follow the usual order of planning, as shown in boldface, and lead directly 
Into evaluation: 

Detcrlptlon of Context 

• What have we accomplished? 
Deicrlptlon of Mlttlon 

• What do we want to accomplish? 
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0«finltion of Goals 



• When do we want to accomplish it? 
Definition of Desired Outcomes 

• What will be the benefits of this accomplishment? 
Strategies and Procedures 

• How can we accomplish it? 

• What resources will we need? 
Evaluation 

• How will we know when we have succeeded? 

t 

These critical questions form the basis of later, more detailed planning that follows the same 
path from initial evaluation of the context through evaluation of the outcomes. Before involving 
representatives of the private sector in planning and evaluation, vocational administrators should 
explore these questions thoroughly to gather as much information as necessary. Once an advisory 
committee or council is formed, it should be treated as a partner in the information-gathering and 
planning processes. 



Planning for Labor Market Needs 

Satisfying the labor marltet needs of both participants in vocational education and employers 
is a primary goal of vocational education. To incorporate these needs in long-range planning, a 
thorough understanding of the labor market context in which the school operates is essential. This 
section highlights the contextual factors that affect labor marltet needs and should be considered 
in long-range planning. 



The Employment Context 

Since local employment is influenced by the broader context, employment data on the 
National and State level should be examined in addition to data on the local level. Obviously, 
National statistics do not describe state or local labor marltets. For example, the unemployment 
rate for a particular month may be 6.5 percent nationally, 10.2 percent in one State, and 3.5 per- 
cent in another. Local labor marltets— the primary concern for local vocational education 
planners— can differ from one another to a great degree, and the local unemployment rate may 
vary considerably from the rate for the State as a whole. It is therefore important to gather as much 
information as possible on the local labor marltet. 

Other aspects of employment should also be considered, such as the special employment 
problems of certain groups, including teenagers, minority groups, and women. In order to plan the 
most useful vocational programs it is best to have data on these groups. Other relevant data to col- 
lect Include the racial and sexual composition of the labor force, the levels of «iducation for those 
working and those not working, the, hiring practices of employers, and average wages and salaries 
for various groups and occupations. 
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It may also be helpful to study specific problems, such as unemployment among high school 
dropouts and the job-hunting techniques of the unemployed. These studies can provide direction 
for long-range planning since they are lil<ely to point to local problems and needs, For example, 
the changing national economic conditions may suggest a need to support local programs for the 
retraining and upgrading of displaced workers. Improvement in the employabllity of youth in both 
inner cities and depressed rural areas, a National priority, may call for specially focused programs. 

Accurate identification of problems such as these requires information. Sources of data 
include National, State, and local employment services, U.S. Bureau of Census surveys and docu- 
ments. Private Industry Councils, and representatives of business, industry, and labor who may or 
may not already be involved in vocational education. A useful aid to gathering information is the 
Occupational Information System (OIS) Handbook (National Occupational Information Coordinat- 
ing Committee 1981), which provides a comprehensive listing and description of labor market 
information for planning needs. Information on National, State, or local priorities is also available 
in several publications. One example is National Priorities for Vocational Education (Ruff et al. 
1981) and Vocational Education: A Look Into the Future (Ruff.'Shylo. and Russell 1981.) 

The Educational Context 

The local labor markat is also defined by the types of vocational education institutions or 
delivery systems available in the area. Vocational education institutions vary in type, program 
adminlsTretlon structure, areas served, governing authorities, relationship to other educational 
agencies and to constituencies, and extent of fiscal autonomy.* These factors reflect different 
objectives, which may be expressed or implied. There are also likely to be differences in the extent 
to which State agencies have an impact, through policy or fiscal provisions, on local vocational 
education institutions. Planning can proceed most effectively when it is based on a clear under- 
standing of the conditions under which the vocational education system operates. 

Initial information gathering should include an examination of all educational and training 
programs in the area in order to avoid duplication of efforts. Iri addition to secondary vocational 
programs, a given area may have a wide variety of offerings, including the following: 

• Private trade and technical schools 

• Private business schools 

* 

• State retraining programs 

• Private junior colleges " 

• Two-year programs in four-year institutions 

• Community-based organization programs 

• Apprenticeship programs 

• Private industry training programs 

• Job Training Partnership Act programs 



'for a detailed description of these factors see Woodruff (197B). 
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Defining the labor market context of local vocational education systems is an important first 
step in the planning process. Vocational program offerings can only be matched to local labor 
market needs when both contexts are fully understood, (For a fuller description of the process of 
gathering and using contextual information, see Copa and Nagi Salem 1982, and Copa and Scholl 
1983.) 



Federally Funded Programi 

As defined In the Education Amendments of 1976 (P.L. 92-482), vocational programs receiving 
federal funds must submit both a long-range plan and a short-range or annual plan based on 
assessment of student and community needs and occupational demand. Each State plan should— 

• establish goals the State will seek to achieve by the end of the long-range period; 

• identify current and projected occupational needs within the State; 

t 

• establish criteria for the coordination of vocational training programs; 

• establish policies and procedures that the State will follow to ensure equal access to 
vocational education programs; and 

• define explicitly the planned uses of Federal, State, and local vocational education funds 
for each fiscal year. 

The requirements for federally funded plans are explicit and demand careful study, More 
detailed information on these plans is contained in Mangum et al. (1979), and Starr et al. (1978). 



Stages In Planning 

Each stage of planning should make use of the information, knowledge, and skills of individu- 
als from business, industry, labor, and education. Planning can be seen as taking place in five 
stages, as depicted in figure 2, chapter 3. 



Stage One: Describe Context and Needs 

Defining the labor market context for the vocational program involves using the data gathered 
In surveying labor market needs. These data should cover employment, social, economic, demo- 
graphic, and political conditions. It is imperative that local members of the private sector partici- 
pate in this stage since their intimate knowledge of the local labor market is invaluable. 



Stage Two: Describe the Program's Mission 

A mission statement simply explains the organization's purpose, or reason for existing. The 
statement should be concise but inclusive, so that it covers all the purposes of the vocational pro- 
gram. From the mission statement, the more concrete goals of the program should logically be 
derived. 



24 



34 



stage Thrt«: Idtnlify Nttdt. Goats, and Objectives 



At this stage, the constraints upon the program are identified. These constraints can include 
financial limitations, timing limitations, policy restrictions, and inadequate human resources. Once 
constraints are identified, their effect on the problem should be analyzed. This can lend to identifi- 
cation of key program needs. Important needs are also likely to have emerged during the first two 
stages. 

When program, needs are agreed upon they should be translated into long-term and short- 
term goals. Long-term goals can be classified in three categories. Client- or product-centered 
goals focus on such specifics as students and curriculum. Administrative-centered goals may deal 
with money, transportation, or fai^lities. Personnel-centered goals focus on teachers, counselors, 
administrators, and other personnel (Dunham 1977). 

Once goafs are listed, they are translated into measurable objectives. Ideally, an objective is 
stated concisely in clear, meaningful terms and deals with only one facet of the problem. It is 
important that each objective be representative of management philosophy and consistent with 
organizational capabilities. 



Stage Four Define the Desired Oulcomet and Benefits 

The benefits of vocational education can be seen as the long-term profits for the community, 
employers, students, and others who are affected by its results (Starr et al. 1982). These benefits 
may be economic, personal, social, and/or educational. Outcomes are the more immediate, or 
short-term, effects of vocational training on former students, business, industry, and labor. Like 
benefits, outcomes can be economic, personal, social, and/or educational. In the process of identi- 
fying benefits and outcomes, business, industry, and labor may contribute distinct perspectives. 

At this point in the planning process it is time to generate alternative methods for meeting 
each objective. By now. those involved in planning have probably been thinking of solutions. 
These can be listed, perhaps during a brainstorming session. Before intensive discussion of solu- 
tions begins, however, the planners should set criteria for selecting methods. These should include 
cost-effectiveness, timing, statements of policy, legislation, and any other criteria the solution 
must meet. Subjective criteria, including such factors as the attitudes of various groups, should 
also be given weight. 

The attempt to weigh solutions objectively against criteria should be seen as a guide to deci- 
sion making— a means of ensuring that all relevant criteria are considered. Decision making 
should not become a purely mechanical process, although structured trade-off techniques are use- 
ful in reducing a large amount of data to manageable proportions. Techniques such as simple 
matrix building with weighting factors or more advanced mathematical tools such as linear pro- 
gramming and goal programming can be helpful in the decision-making process if all parties 
involved are comfortable with this method. 



Stage Five: Describe Procedures for Implementation 

/ 

The final stage of the planning process is the synthesis of the selected solutions into an overall 
plan. The plan should in^ude specific procedures tor interagency coordination. Successful inter- 
agency coordination no/only facilitates the meeting of objectives, but bears dividends by leaving a 
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good Impression with business, industry, and labor representatives and local political interests 
(Starr et al. 1983). When vocational education systems are coordinated, they are viewed by the 
community as more useful and more relevant. 

Successful planning requires understanding the necessary stages in the planning process. 
Equally important are the interrelationships among the groups involved in planning. Finally, good 
planning must rest on a base of information. Although the process described here taltes time, it 
can be used efficiently to arrive at solutions that represent the best efforts of vocational education, 
business, industry, and labor working together. The solutions generated through this process have 
a far better chance of working effectively to improve the vocational education program. 
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CHAPTER 5 
EVALUATING TOGETHER 



A Program Evaluation Mddal 

During the evaluation process It Is Important to bear In mind the large context in which evalua- 
tion and planning take place. Evaluators should examine and consider socioeconomic and demo- 
graphic data, labor market factors, and educational factors, such as the types of local vocational 
education institutions and delivery systems and the system of educational governance. 

A comprehensive model of program evaluation. Including three evaluation processes. Is Illus- 
trated In figure 3. One type of evaluation shown in this model is needs assessment, used in the 
initial phases of a program to make planning decisions. Process evaluation is used while a pro- 
gram is operating to provide information for making implementation decisions. A useful technique 
for process evaluation is time-on-task assessment (see Halasz and Desy 1984). Product evaluation 
is usually done at the end of a program or whenever an outcome has been identified. This type of 
evaluation is used in making decisions about the future of a program or its components (McCaslin 
n.d.). 



Perspectives on Evaluation 

For some people the term "evaluation" has negative connotations, suggesting a generally crit- 
ical judgment. In reality, educational-evaluation is intended to be a positive activity, involving the 
use of inforrnation^to nriaketlecisions about a program (Cronbach l983).The goal of evaluatJon Is 
to judge and improve the worth of an educational program (Stuff lebeam and Webster 1983). 

Evaluation can be approached from numerous perspectives. Stufflebeam and Webster classify 
evaluation studies into three main approaches, each with ltd own perspective and each based on 
distinct kinds of studies. These major perspectives are the politically oriented, the question 
oriented, and the values oriented. They are shown with the types of study associated with them in 
figure 4, and may suggest appropriate kinds of study to support a particular evaluation. Stuffle- 
beam and Webster (1983) give a detailed description of these approaches. They conclude that no 
one approach or type of study is consistently the best for evaluating education, but that planners 
must select the approach that best fits their needs. 

In addition to a group's overall perspective, individual perspectives will be brought to planning 
and evaluation by those involved. The administrator, the curriculum specialist, the teacher, the 
guidance counselor, the advisory council member, and the employer all approach evaluation 
according to their individual roles and functions. The process of evaluation must recognize and 
respect these individual perspectives. 
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NEEDS ASSESSMENT 



r 



• Individual Needs 
• Interests 
^ Knowledge 

• Cognitive 

• Psychomotor 

• Affective 



* Employer Needs 

• Industrial Growth 
9 Occupational 

Supply/Demand 

• Present 

• Projected 



Societal Needs 

• Educational 

• Economic 

• Social 

• Government 



PROCESS EVALUATION 



1. 



y 





• 


Instructional Programs 






• Content 






• Sequence 






• Delivery Modes 






• Time on Task 




• 


Personnel 






• Qualifications 






• Iriservice 






• Equity 






• Time on Task 




• 


Students 






• Aptitudes 






• Interests 






• Performance 






• Time on Task 




• 


Support Services 






• Guidance & Counseling 






• Career Exploration 






• Job Placement 




• 


Community Resources 






• Advisory Councils 






• Business, Industry, 






Labor Linkages 






• Public. Relations 




• 


Facilities & Equipment 






• Quantity 






• Quality 






• Relatedness 



PROD UCT EVALUATION 

i 



Student Technical Competencies 

• Former Student Job Roles 

Outcomes 
Student Nontechnical Competencies 

• Former Student Nonoccupational 

Outcomes 



I 
I 
1 
I 



I 



Societal Outcomes 

• Economic 

• Psychosocial 

• Educational 



Figure 3. A compreheniive program tvaluation modtl 



SOURCE; Adapted from McCaaiin (n.d.. n.p.). 
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PERSPECTIVE 



TYPES OF STUDY 



^ 

Politically oriented j politically controlled j 

I public relations inspired i 

L J 



Question oriented 



objective based 
accountability 
experimental research 
testing programs 
management information 
system 



Values oriented 



accreditation/certification 
policy oriented 
decision Of iented 
consumer oriented 
client centered 
connoisseur based 



Figure 4. Evaluation pertpectlvet and typei of itudy 



: SOURCE: Adapted from Stuff lebeam and Webster (19B3. pp. 25-35). 
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Evaluating tha Vocational Education Plan 



Critarla 



Once representatives of business, industry, labor, and education have developed a plan, the 
plan should be carefully evaluated in light of criteria developed by the planner;. According to 
Dunham (1977), whatever other criteria are used, the plan should be— 

• accurata— based on good data and information: 

• fartighted— addressing needs and issues that will probably be important for some time to 
come; ' 

f rtasonable— taking into account available and projected resources; 

• broad-based— involving many people and issues; 

• systematic—moving from the present to the desired objectives in a well-defined and effi- 
cient way; 

• useful— possessing benefits for everyone who participates in the process; 

• responsive- reflecting real needs, issues, and problems. 

The remaining criteria for evaluating the plan can be seen as steps in evaluation that grow 
from the planning process. 

stages of Evaluation 

The first step in evaluating a vocational education plan is to reassess the context of the voca- 
tional program. The major questions to ask are related to evaluation tasks. They fall in four general 
categories. 

r ■ - 

Datcription of Contaxt 

• What is the labor market context for this program? 
« How does this plan take that context into account? 

, I 

Datcription of MiMlon I 

• What are our overall instructional purposes? 

• What are our labor market intents? 

• Who is to receive training? 
Raassattmant of Goals 

• What program needs have to be considered if the mission is to be fulfilled? 

30 
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R«Miettm«nt of Dttired Outcom«' 



• What outcomes and benefits will serve as a basis for evaluating the success of the plan? 



Ftdf rai Evtluation Raquiramentt 

As a result of the passage of the Education Amendments of 1976 (P.L. 94-482). evaluation of 
vocational education is now required. State and local education agencies, their boards of educa- 
tion, and their advisory committees or councils are given evaluation responsibilities under this act, 
which requires the involvement of business, industry, and labor. According to Jennings (1979), 
Congress's primary purpose in establishing the evaluation requirements was to gi^e local educa- 
tors and State administrators an opportunity to learn how their programs are operating, and thus 
to encourage improvement of those programs. 

The amendments require vocational educators to determine program quality according to 
three criteria: effectiveness, efficiency, and relevance. Section 112(b) (1) (A) of the amendments 
states that these evaluations shall focus on the following: 

(a) Planning and operational processes such as: 

(1 ) quality and availability of instructional offerings; 

(2) guidance, counseling, and placement and follow-up services; 

(3) capacity and condition of facilities ai, equipment; 

(4) employer participation in cooperative programs of vocational education; 

(5) teacher/pupil ratios; and 

(6) teacher qualifications. 

(b) Results of student achievement as measured, for example, by: 

(1) standard occupational proficiency measures; 

(2) criterion-referenced tests; and 

(3) other examinations of students' skills, knowledge, attitudes, and readiness for 
entering employment successfully. 

(c) Res>'*^ of .ludent employment success as measured, for example, by: 

(1 ) ft tes f employment and unemployment; 

(2) rzg^i rates; 

(3) duration of employment; and 

(4) employer satisfaction with performance of vocational education students as com- 
pared with performance of persons who have not had vocational education. 
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(d) The results of additional services as measured by the suggested criteria under para- 
graphs (a), (b), and (c) o' this section, tf iat the state provides under the Act to these spe- 
cial populations: 

(1) women; 

(2) members'. :>f minority groups; 

(3) har^dlcapped persons; 

(4) disadve" tage J pers ons; and 

(5) rei'sons of limited English-speaking abiHt/. 

Meeting ihosa requirements, espe«i;ially those in r^arayraph (c) demands the involvement of 
business, industry, and labor in this evaluation process. The strateg.os for involvement discussed 
In this handi>5ok an be employed to gain an J use the input from representatives of the private 
sector most effecti, ely. 

Evaluation is an iniegral part of the planning process. An effective plan is based on extensive 
evaluation of the context, needs, resources, and so on of the vocational education program. When 
the plan is formed, it sho<jld be evaluated, as discussed earlier. As outcomes are realized, or 
phases of the plan are completed, evaluation of effectiveness is in order. 

Throughout the evaluation process, representatives of business, industry, and labor should be 
Involved in appropriate ways. The opportunity to aid in Judging the value of the plan or program Is 
one most people take seriously and see as an honor. Inviting representatives of the private sector 
to help with the process of evaluation not only draws upon the knowledge of these individuals, but 
also helps cement their involvement in the vocational education program. 
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CHAPTER 6 



WORKING TOGETHER: 
TECHNIQUES FOR EFFECTIVE INVOLVEMENT 



The identification of techniques for involving business, industry, and labor in vocational plan- 
ning and evaluation should flow from the planning and evaluation tasks. The process begins with a 
determination of priorities and moves on to the selection of tasks and choice of a strategy, as 
described in chapters. Finally, specific techniques are chosen to implement the selected strategy. 

The following figures and worksheets are a guide to the process of selecting techniques. Fig- 
ures 5 and 6 illustrate this process in terms of specific planning and evaluation tasks. Worksheets 2 
and 3 are forms for use in choosing techniques through the same process, which moves from task 
to strategy to technique. 
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PLANNING TASK 



Describe the context and needs for vocational education. 



STRATEGY 



Increase communication. 



Clarification Statement: This will involve determining public opinion regarding the 
needs of vocational education. An information exchan^^o network will be estab- 
lished to collect relevant and timely information on the context and needs for voca- 
tional education in this labor market area. 



TECHNIQUE 



Delphi 



Brainstorming 



Discussion 



Figure S. Stepi for lelectlon of planning atrateglet and techniques 
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EVALUATION TASK 



Reassess program needs, goals, and objectives. 



STRATEGY 



Provide mechanisms for joint decision mal<ing. 

Clarification Stattmtnt: This will involve the establishment of a goal identification, 
assessment, and selection committee. Group sessions of five business representa- 
tives, five industry representatives, five labor union representatives, and four educa- 
tion representatives will be conducted. A survey of local business, industry, and 
labor representatives will be conducted to provide Input for the group sessions. 



TECHNIQUE 

• Nominal group technique 

• Mail survey 

Figure 6. Steps for ••lection of ovaluation «trategi^s and t^chniqu^s 
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WORKSHEET 2 

SAMPLE FORM FOR SELECTION OF 
PLANNING STRATEGIES AND TECHNIQUES 



PLANNING TASK 



STRATEGY 

ClarKlcallon Statement: 



TECHNIQUE 



ERIC 
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WORKSHEET 3 ^ 

SAMPLE FORM FOR SELECTION OF 
EVALUATION STRATEGIES AND TECHNIQUES 



EVALUATION TASK 



STRATEGY 



Clarification Statement: 



TECHNIQUE 
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Figure 7 indicates some of the functions specific techniques serve. Many techniques can also 
be used in ways hot shown. 

Each description of a technique includes— 

• a definition, 

• an explanation of how to use the technique, 

• its positive and negative attributes, and 

• references to documents that provide a detailed description of the techniq|je. 

Much of the value of any technique depends on how it is used, so that a guarantee of results is 
Impossible. The sidll of the user, the context of the vocational education system, and the intended 
outcome all influence the effectiveness of a technique. Used properly, these techniques are excel- 
lent tools with the capability of involving people of diverse interests in the common goal of improv- 
ing vocational programs. 

The techniques are given in alphabetical order with no implication as to relative effectiveness. 
Both the manner of presentation and many of the concepts were derived and adapted from Hull 
and McCaslin's (1977) CBreer Education ImpiBmentatlon: A Handbook for Strategy Development. 

These techniques wero selected because they seemed particularly useful in. working to estab- 
lish a council. There are many other techniques that may be more suitable for a specific situation. 
Readers should freely explore other methods. In addition, ways to adapt the techniques included 
here should be explored. 
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FUNCTION 



TECHNIQUE 



Analyzing 



Enhancing communication 



Gathering Information 



Generating ideas 



Involving others 

Presenting information 
Problem solving 



Setting goals 



audit 

cause and effect analysis 

brainstorming 

discussion 

DACUM 

process cause and effect 
Nominal group technique (NGT) 
quality circle 

discussion 
quality circle 

audit 

Delphi 

survey 

brainstorming 
Delphi 
discussion 
NGT 

quality circle 
Delphi 

quality circle 

illustrated talk 

cause and effect analysis 

discussion 

NGT 

process cause and effect 

brainstorming 

discussion 

NGT 

process cause and effect 



Figure 7. A guide to the uses of telecied techniques 



SOURCE: Adapted from Hull and McCaslin (1977). 
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AUDIT 



An examination of existing program records in order to verify results, 
reveal possible weaknesses in procedures, or identify problems. 

DMcription 

An audit is conducted by an independent auditor or team of auditors. Audits have in common 
a four stage process which may be adjusted to the scope and objectives of a specific audit. The 
steps are given here, although the process is one which should be studied more fully before it is 
implemented. 

• Prepare for the audit 

—Selection of the audit team 
— Pre-engagement contact 
—Auditor familiarization 
—First draft of the audit plan 
—Audit entrance conference 
—Walk-through 

—Revision of the audit plan " 

• Conduct the audit 
—Information gathering through interviews, observation, or review 

• Report on the findings 

—Ongoing reports to the auditee 
—Flash reports to the audit recipient 
—Draft report of findings 
—Audit exit conference 
—Final report 

• Reconcile findings with audit recipient and auditee 

—Review of findings 
—Joint agreement on a plan of action 
—Review of corrective action 
—Follow-up audit 

.. 40 
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DitcuMlon 

The findings of an audit may support continuation of current efforts or provide impetus for 
change. In either case, they are ideally used to facilitate policy planning and management. Prob- 
lems may result when audit findings are misused or simply disregarded. 

Recommendations following an audit are more likely to be implemented when they consume 
minimal time and money, are flexible and present alternatives, call for incremental rather than total 
change, provide staff with a central and reasonable role, and are concise and easily 
communicated. 

Positive Attributti 

• Implies a degree of accountability for compliance to accepted standards 
e Aids management in efforts to improve future operations 

• Improves understanding 

• Provides an opportunity to gather information 

Negative Attributes 

e May cause auditee to feel threatened 

• Can result in development of mistrust 

• Can be difficult to locate a competent auditor 

• May lead to misuse of results 

Related Readings 

Adams, Kay A., and Walker. Jerry P. Improving the Accountability of Career Education Programs: 
Evaluation Guidelines and Checklist. Columbus: The National Center for Research in Voca- 
tional Education, The Ohio State University. 1979. 

Stephens. Nancy F. Evaluation Guidelines and Practices for State Advisory Councils. Columbus: 
The National Center for Research in Vocational Education. The Ohio State University, 1980. 
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BRAINSTORMING 



A technique for developing a large number of creative ideas in a short 
time. 



Ottcription 

Brainstorming is most effective when confined to simple, uncomplicated problems or ques- 
tions. Participation is highest when the group includes about 12 people of approximately the same 
rank. Each member offers one idea in turn; ideas are not evaluated, discussed, or critiqued. A 
quantity of free wheeling ideas are sought, and combining or improving ideas is permitted. The 
optimal length for a brainstorming session is 30 minutes. The steps in brainstorming include the 
following: «, 

• Define a specific topic for the session. 

• Review guidelines for brainstorming. 

• Commence brainstorming, recording ideas as they are given; the exhaustion of ideas sig- 
nals completion.. 

• Vote on the ideas generated, circling those that receive the most votes. 

• Discuss the ideas that receive the most votes and rank them in order of perceived 
importance. 



. DitcuMion 

Brainstorming is useful during many phases of planning, such as defining needs, generating 
goals and objectives, or finding ways to implement solutions. It is a supplemental technique that 
can be used with both basic and process cause and effect analysis, surveys, discussion, and so on. 
Brainstorming is based on the principles that one thought leads to another and that more Ideas are 
generated by a group than by the same individuals working alone. 



Positive Attributes 

• Results in a permanent record of possibilities 

• Promotes equal participation 

• Leads to piggybacking of ideas 

• Is useful in most phases of problem solving 
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Negatlvt Attributtt 

• May produce a "bandwagon" effect 

• May be dominated by one or two individuals 

• Requires time 

• Requires that group members have l<nowledge of a given topic 
R«lattd Readings ■ ^^r. 

Larey. Will. "Brainstorming: Guidelines for Generating Ideas." Training 19. no. 9 (September 1982): 
114-115. 

Nickens. John M.; Purga, Adalbert J. Ill; and Noriega, Penny P. Research Methods for Needs 

Assessment. Washington, DC: University Press, 1980. 1 

Rawlinson, J. G. Creative Thinking and Brainstorming. New York: Halsted Press, 1981. 

Osborn, A. Applied Imagination. New York: Creative Problem Solving Institute, University of 
Buffalo, 1962. 



43 

53 



CAUSE AND EFFECT ANALYSIS 



A graphic method for identification of the potential causes of a problem. 



Description 



This method calls for construction of a "fish bone diagram" as the group brainstorms the pos- 
sible causes of a problem. A statement of the problem is written to the right on an easel pad and 
the possible problem groupings— materials, personnel, methods, and materials— are written to the 
left. The group is likely to incorporate other specific problems groupings as they brainstorm. Foh 
lowing the brainstorming, cruse and effect iinalysis is used to identify the most likely causes. The 
action steps are as follows: 

• Develop a precise statement, of the problem. 



• Identify and record major causes by category. ' 

• Select the most significant causes. 

• Rank the likeliest causes in order of their probable importance through voting (one vote ' . 
per member). 

• Verify those causes designated as most likely to have created the problem. 

• Prepare a recommendation for action. 



The group should work with its own problems, not those belonging to others, when selecting 
the problem from within its area of work responsibility. The problem or effect may be one identi- 
fied during an earlier brainstorming session. Cause and effect analysis attempts to explain why the 
problem or effect exists, and can be used to find the cause of a favorable outcome. In examining 
possible causes the group should pay particular attention to deviations from the norm and recur- 
ring patterns. 



Positive Attributes 

• Easy to use 

• Promotes participation 

• Promotes probi ^m solving rather than fault finding 



Diicuision 
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Ntgatlvt Attrlbutct 

• User skill needed 

• Problem mv be wrongly lder)tifled or Imprecisely stated 

• Can require considerable time 

References 

Bellanca, James. "Quality Circles: Making Schools Productive." VocEd (May 1982): 31-33. 

Dewar, Donald L. Quality Circle Member Manual. Red Bluff, CA: Quality Circle Institute, 1980. 

Mohr, W. L., and Mohr. H. Quality Circles: Changing Images of People at Work. Reading, MA: 
Addlson-Wesley, 1983. 
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DACUM (Developing A Curriculum) 



An approach to identifying competencies or tasks that must be oer- 
formed by persons in specific occupational areas. 



Description 

A group technique. DACUM involves the processes of analysis, charting, rating, selection, and 
sequencing. It is graphic in nature, presenting definitions of an entire occupation on a sheet of 
paper. It is considered to be a form of occupational analysis rather than a curriculum evolving from 
an analysis. An expert DACUM facilitator is needed to conduct the session. 

The DACUM committee consists of 10 to 12 experts in an occupational area who work for 3 
days to develop the DACUM chart. These committee members are frorri business, industry, or the 
professions, and do not include vocational educators. The general aroa^ of competence required 
in the occupation are defined and each is subdivided into those individual tasks that collectively 
enable an individual to perform competently within that general area. Efich task can later serve as 
an independent goal for learning achievement. The committee follows these steps: 

• Provide introduction and orientation. 

• Review a description of the specific job or occupational area. 

• Identify the general area of competence. 

• Identify the specific skills for each area of competence. 

• Review, refine, and sequence the skill statements. 

• Establish levels of compb.tjice for each skill. 

• Structure the competency profile chart at the end. 



Positive Attrlbutei 

• Serves as a flexible tool for occupational and subject analyses 

• Involves workers from the occupation who can represent the area accurately 

• Results are useful for multiple purposes 
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Ntgatlvt Attributti 

• Cannot be conducted without a trained facilitator 

• Is time-consuming 

Rtftrtnc«t 

Sinnett. W. E. The Application of DACUM in Retraining and Postsecondary Curriculum Develop- 
ment. Toronto, Ontario: Number College of Applied Arts and Technology, Randa Division, 
1974. 

Buchar, F. J. "DACUM Dialogue: Answering Basic Questions." CTM: The Human Element 14. no. 5 
(February 1982): 20-24. 

Norton. R. DACUM: Developing a Curriculum. Columbus: The National Center for Research in 
Vocational Education, The Ohio State University, 1979. 
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DELPHI TECHNIQUE 



A reliable way to obtain the consensus of a group of persons with exper- 
tise in a field. 



Oetcriptlon 

The Delphi is essentially a series of questionnaires, interspersed with information and sum- 
maries of feedback from earlier questionnaires. Each questionnaire is built upon responses to the 
preceding questionnaire, and the rounds of questioning stop when consensus is neared or suffi- 
cient information has been obtained. The final analysis considers the information gathered from all 
rounds of the Delphi. Consensus is not measured by percentage of votes in a prescribed range, 
but by the stability of respondents' vote distribution curve over successive rounds. A marginal 
change of less than 15 percent defines stability. Delphi participants need not meet face to face. 
There can be several panels of participating experts; each should have a least seven members. The 
steps in conducting a Delphi include the following: 

• Designate a person or persons to conduct and monitor the procedure. 

• Involve people with expertise in the field. 

• Develop the first questionnaire internally, or by requesting questions from experts regard- 
ing the topic to be studied. 

^ • Send the fir^t questionnaire to the panelists. 

• Analyze first-round responses; when a participant's response differs from the consensus, 
request further information or justification for this point of view. 

• Prepare and transmit a second-round questionnaire containing feedback on the results of 
the first round. 

• Analyze the second-round responses. 

• Conduct further rounds if desired. 

• Analyze the overall results. 

• Report the findings to panelists. 



DItcuttlon 

The Delphi is useful when individual judgments must be tapped and combined, whtn there is 
disagreement, and when knowledge is incomplete. It can be used to define either the nature of the 
problem or the components of a successful solution. It is a problem-solving or idea-generating 
strategy. 
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Potltivt Attributes 



• Is relatively low in cost 

• Does not require participants to meet 

• Avoids domination by a single personality, so that a "bahdwagon" effect is unlikely 

• Generates ideas and solutions 

• Yields a pooled judgment 



Negative Attributes 

• Is time consuming 

• Can reflect the typical pessimism of long-range forecasts and undue optimism of short- 
range forecasts 



References 

I 

Delbecq, Andre L.; Van de Ven, Andrew H.; and Gustafson, David H. Group Techniques for Pro- 
gram Planr)ir)g. Glenview. IL: Scott, Foresman, 1975. 

Helmer, Olaf. Analysis of the Future: The Delphi Method. Santa Monica, CA; The Rand Corpora- 
tion, March 1967. 

Hopkins, Charles O.; Ritter. Kenneth L.; and Stevenson, William W. Delphi: A Planning Tool. 
Stillwater; Division on Research, Planning, and Evaluation, Oklahoma State Department of 
Vocational and Technical Education, January 1973. 

Lewis. Morgan, and Russell, Jill F. Trends, Events, and Issues Likely to Influence Vocational Edu- 
cation in the 19d0s. Columbus: The National Center for Research in Vocational Education, 
The Ohio State University, April 1980. 

tinstone, Harold A. "The Delphi Technique." In Handbook of Futures Research, edited by Jib 
Fowles, Westport, CT: Greenwood Press, 1978. 

Parker. James T., and Taylor, Paul G. The Delphi Survey. Belmont, CA: Fearon Pitman, 1980. 
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DISCUSSION 



A technique for promoting the verbal exchange of ideas and perspectives 
on selected topics. 



Description 

A discussion should be based on accurate information and facts rather than uninformed opin- 
ion and emotion. To be productive, a discussion requires thoughtful planning and leadership. Use 
ful steps in conducting a discussion include the following: 

• Prepare a list of pertinent leading questions. 

• Introduce a topic; define the limits of the topic, and set the time schedule. 

• Promote participation unobtrusively, clarifying the problem, defining new terms, correct- 
ing misunderstandings and misinterpretations, and helping participants express their 
ideas. 

• Summarize periodically. If appropriate, play devil's advocate. 

• Draw conclusions, formulate consensus view, and suggest ways to improve the 
discussion. 

• Lead group in deciding whether further action is advisable. 



DItcuttlon 

Topics for discussion should grow from planning and evaluation activities and should be part 
of the process of forming and exploring new relationships between ideas and data. Discussion is 
useful for improving problem solving and group communication, and can be used to lead directly 
to definite group decisions and goal setting. ^ 



Poiltlve Attributei 

• Exchanges and tests ideas 

• Stimulates thought and analysis 

• Promotes active involvement, a sense of ownership, and perceived relevance on the part 
of participants 
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N«gativ« AHributM 



• Is not ideal for presenting information 

• Consumes time 

• May lead to an unbalanced presentation of various points of view 

• I nvolves more talk than action 



Further Readings 

The National Center for Research in Vocational Education. Conduct Group Discussions, Panel 
Discussions, and Symposiums. Module C-17 of Category C. Instructional Execution Profes 
sional Teacher Education Module Series. Athens, QA: American Association for Vocational 
Instuctional Materials, 1977. 
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ILLUSTRATED TALK 



An oral presentation supplemented with illustrative aids, used to convey 
a body of information to a large group. 



Oeteription 

An illustrated talk is useful in large group situations where individual attention and interaction 
are impractical. This method is particularly suitable when the audience knows little about the 
topic, since the presenter is able to. synthesize information from several sources into an organized 
presentation. The illustrated talk is most effective when the presenter possesses public speaking 
skills and the audience is able to listen attentively and possibly take notes. The steps of an illus- 
trated talk include thq following: 

• Plan the presentation, including purpose, introduction, key points, summary, and 
evaluation. 

• Be thoroughly familiar with your material. 

• Deliver the presentation: use public speaking sK .<s such as— 

good audience contact, 

an enthusiastic manner, 

clear, audible speech, 

gestures for emphasis, and 

a conversational and natural style of delivery. 

• Follow the presentation with discussion, open questions, or another activity that will indi- 
cate how well the information was communicated. 



DItcuttlon 

The talk should be planned around the audience rather than the material. It should have rele- 
vance for the attending group by virtue of its focus on issues that have been or will be covered in 
the planning and evaluation process. The illustrations used to clarify concepts can be verbal, vis- 
ual, or both. Useful verbal illustrations include analogies, frames of reference, and examples. Vis- 
ual illustrations can be graphs, diagrams, slides, and so on; the use of audiovisual techniques 
demands thorough familiarity with the equipment and careful preparation. 
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Poiitivt Attrlbuttt 



• Can convey information well 

It 

<• • Can be tailored to the audience 
Negative Attributes 

• Demands good public-speaking sl(ills; otherwise, may lack liveliness and spontaneity 

• Promotes more interaction with the instructor than with the information 

• Creates difficulty in monitoring audience comprehension 

Related Readings 

The National Center for Research in Vocational Education. Present an Illustrated Talk. Module 
C-15 of Category C. Instructional Execution Professional Teacher Education Module Series. 
Athens, GA: American Association for Vocational Instructional Materials, 1977. 
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NOMINAL GROUP TECHNIQUES (NOT) 



A process for gaining consensus on a problem or issue, developing solu- 
tions, and identifying priorites. 



DMcriptlon 

This is a procedure for obtaining consensus on a problem or issue, develofSing solutions, and 
identifying priorities. The NGT consists of generating, listing, discussing, and voting on responses 
to a predetermined question. The f lal vote combines individual judgment into a group decision or 
consensus. 

An NGT session begins with a clear, concise definition of the NGT steps and time for ques- 
tions and answers about the process. The steps continue as follows: 

• PhiMl 

Clearly worded task statement is presented. 
Participants write responses (10 minutes). 

• Phase 2 

Each member states one response (or chooses to pass); there is no discussion. 
Responses are recorded on a large easel pad. 
Members augment their lists if they choose. 

• Phases 

Items are clarified without evaluation. 

• Phase 4 

Each participant ranks all items in writing, with the lowest being number one. 
During a break, result.* are tallied and recorded on the easel pad. 
Results are discussed. 

Discussion 

The ideal NGT group has between 5 and 12 participants. Larger groups should be divided into 
groups of this size and concurrent sessions held with one large summary session at the conclu- 
sion. The process can be completed in 90 minutes to 2 hours. 
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Potitivt Attributes 

• Gives group members an equal voice 

• Is problem solving and goal oriented 

• Depersonalizes ideas 

• Accommodates several ideas at one session 

• Tolerates conflicting ideas 

• Promotes piggybacl(ing of ideas 

• Provides a mechanism for participation and involvement of group members 

Negative Attributes 

• Not suited to less-involved or less-complex problems and discussions 

• Time consuming 

• Requires a leader with skill, confidence, legitimacy 
References 

Oelbecq, A.; Van de Ven, A.; and Gustafson, J. Group Techniques for Program Planning. New 
Yorl<: Scott, Foresman, 1975. 

Coke, J. G., and Moore. C. M. Guide for Leaders Using Nominal Group Technique. Columbus, OH: 
Academy for Contemporary Problems, 1978. 
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PROCESS CAUSE AND EFFECT ANALYSIS 



A technique for working fronn a problem to its cause In order to see 
where a breakdown may have started. 



Description 

This method is distinguished from basic cause and effect analysis by its analysis of a 
sequence of steps. The problem statement is written to the right on an easel pad or chalkboard 
and the steps in the process are broken down and written as "blocks" leading to the problem. Each 
step of the sequence should be fully brainstormed. The steps are as follows: 

• Identify the problem as precisely as possible. 

• Determine the process sequence— the blocks— by working either backward from or for- 
ward to the problem. 

• Depict the sequence graphically with cause-effect Illustration. 

• Brainstorm on possible causes, recording the ideas generated. 

• Examine all ideas and vote on those considered most important (multiple votes per per- 
son are allowed); record the vote. 

• Circle the two or three causes receiving the most votes, and rank in order of importance. 

• Discuss each cause and take a final vote (one. vote per participant). 

• Record the vote and rank possible causes according to votes received. 

• Verify the identified major cause. 

• Make recommendations and present the chosen solution. 

J 

Difcuttlon 

Process cause and effect analysis is advantageous in cases where a sequence of steps does 
lead to the effect. Identifying the blocks in the sequence often lends insight to the diagnostic pro- 
cess and can ultimately save time by pinpointing the source of a problem. 

Positive Attribute t 

• Promotes participation 

• Promotes problem solving rather than fault finding 
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Negativt Attributts 

• Requires skill in using the method 

• Requires precise identification of the problem 

• is inaccurate if an erroneous process sequence is established 

Reiatvd Readings 

Bellanca, James. "Quality Circles: Making Schools Productive." VocBd (May 1982): 31-33. 
Dewar, Donald L. Quality Circle Member Manual. Red Bluff, CA: Quality Circle Institute, 1980. 
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QUALITY CIRCLE 



A group process to bring about improved work environments and pro- 
ductivity with total work force involvement. 



ERIC 



Description 

This method is used within a defined work environment, and can be instituted with the teach- 
ing and/or administrative staff within a vocational education system. Five to ten employees from 
one area meet regularly and voluntarily to identify, analyze, and solve problems. The basic steps in 
the process are as follows: 

• Identify voluntary participants. 

• Participants e.stablish goals. 

• Participants areirained in problem solving. 

• Group selects a facilit^or 

• Group analyzes and prioritizes problems. 

• Team responsibilities for implementing recommendations are defined. 
Discussion 

This process brings together a group of workers who volunteer to share responsibility for a 
function or task in order to produce a product or service. Quality circles usually meet to identify, 
analyze, and generate solutions to priority problems defined by the group. 

Ideally, group memoers are given instruction in group communications, interpersonal skills, 
problem-defining and problem-solving strategies, motivation, and leadership skills. The measure- 
ment techniques and quality strategies also taught to this group may include check sheets and 
graphs, cause and effect diagrams, pareto diagrams and histograms. For advanced quality circles, 
more advanced measurement and quality strategies are taught, such as sampling methods, control 
charting, and other methods. 

Positive Attributes 

• Provides opportunity for staff involvement 

• Encourages employer-employee participation on critical work environment -^roblems 

• Assists in improvement of productivity 
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• Addresses queations of cost reduction 

• Provides basis tfor improving employee self-esteem 



Ntflatlve Attributii 

• Requires weekly meetings; can be time consuming 

• Increases costs in terms of time spent away from production 

• Creates additional responsibilities for pi iinel 

• Requires additional training foV members 



Refertncet 

Harris. P. R. New World, New Ways, New Management. New York: Af^ACON, 1983. 

Harshman. Carl L. Quality Circles: Implications for Training. Columbus: The National Center for 
Research in Vocational Education, The Ohio State University, 1982. 

Lloyd, Russell F., and Rehg, Virgil R. Quality Circles: Applications in Vocational Education. 

Columbus: The National Center for Research in Vocational Education, The Ohio State Uni- 
versity, 1983. 

Maccoby, M. The Leader: A New Face for American Management. New York: Simon and Schuster, 
1981. 

Mohr, W. L.. and MOhr, H. Quality Circles: Changing Images of People at Work. Reading, MA: 
Addison-Wesley Publishing Co., 1983. , 

Ouchi. W Theory Z: How American Business Can Meet the Japanese Challenges. Reading, MA: 
Addison-Wesley Publishing Co., 1981. 

Yankelovich, D. New Rules: Searching for Self-Fulfillment in World Turned Upside Down. New 
York: Random House, 1981. 
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GURVEY 



A procedure for gathering information about a population with respect to 
a given area of Concern or question. 



Description 



This method typically collects data from a sample of a population, most commonly through 
questionnaires, or through direct or telephone interviews. The representativeness of a mail survey 
depends heavily on the clarity and quality of the data-gathering instrument completed by the 
respondents. In telephone surveys the quality of the interviewers, who must interact with respon- 
dents and accurately record all information, is very important. Steps in a survey include the 
following: 

• Formulate the problem and the informati'on needed. 

• Determine the question(s) to be asked and develop or adopt a data-gathering instrument. 

• Designate the population to be surveyed and/or select sample respondents. 

• Specify procedures for data collection. 

• Collect data and report findings. 



Surveys can be an inexpensive procedure for gathering information, as well as a means of 
communicating with large numbers of individuals. Tables are available for the determination of a 
sample size that will provide an acceptable level of accuracy. Surveys do not need to be large or 
complex. Cooperation with local agencies or institutions before the initiation of the survey can 
contribute to the study design and to the resources available to conduct the work. 



PoBitive Attributes 

• Gathers a large body of data 

• Provides insight into how a population views a problem or question 

• Is relatively low in cost 



Diicusslon 
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Ntflatlvt Attribute 

• Is time consuming 

• Requires expertise in questionnaire design, data processing, and analysis 

• May conclude with an insufficient response rate 

Related Readings 

Dillman, Don A. Mail and Telephone Surveys. New York: John Wiley and Sons, 1978. 

Nickers, John M.; Purga, Adelbert J.. Ill; and Noriega, Penny P. Research Methods for Needs 
Assessment. Washington, DC: University Press of America, 1980. 

Nunez, Ann P., and Russell, Jill F. As Others See Vocational Education, Book 2: A Survey of the 
National Conference of State Legislatures. R&D Series No. 225B. Columbus: The National 
Center for Research in Vocational Education, The Ohio State University, 1982. 

Franchak, S. J., and S niley, J. Evaluating Employer Satisfaction. Columbus: The National Center 
for Research in Vocational Education, The Ohio State University, 1981. 

Ponce. E.. and Franchak, S. J. Evaluating Student Satisfaction. Columbus: The National Center for 
Research in Vocational Education, The Ohio State University, 1981. 
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Appendix A 
Analysis of Existing Data 



Existing data or information were used to conduct secondary analysis regarding the percep- 
tions and conditions surrounding business, industry, and labor involvement in the evaluation and 
planning of vocational education. Several recent studies (Coleman and Crowe 1961; McKin ,ey et 
al. 1981; Nunez and Russell 1982) have included questions that relate generally to the involvement , 
of business, industry, and labor in vocational education. Although specific evaluation and planning 
issues were not the focus of these studies, it was apparent that some of the questions could help 
illuminate the problem addressed by this study. Existing databases from previous studies relating 
to business, industry, and labor linltages were the primary sources for the secondary analysis of 
data. For example, efforts in business-labor-education collaboration have been conducted by the 
National Institute for Worl( and Learning (Elsman 1981) and the National Association for Industry- 
Education Cooperation (Clark 1962). Other organizations working to facilitate collaboration are 
the Chamber of Commerce of the United States, the National Association of Manufacturers, and 
the AFL-CIO Department of Education and Human Resources Development Institute. 

Open-ended discussions with State research coordinating unit (RCU) directors and State and 
local evaluation and planning coordinators were held to identify evaluation and planning studies 
that were not included in the normal stream of information dissemination. These individuals were 
also asked to nominate exemplary efforts in their respective states that involve employers and 
labor representatives and vocational educators in planning and evaluation. Criteria used in this 
process included the following: 

• A minimum of five vocational education program areas in the local education agency 

• Documented evidence of business, industry, and labor involvement in local education 
program reviews and local program plan development 

• Local education agencies located in (1) a central city with a metropolitan population over 
500,000, (2) a city or town with a population of 25,000-100,000, (3) a town or region with a 
population below 25,000 

• Willingness of local education agencies and industries to participate in the study 

• Consideration of task constraints, such as level of funds, staff, and time 

Information was secured about these efforts by telephone conference calls and mail corre- 
spondence. Information was also collected at working sessions of meetings of the annual confer- 
ences of the American Association for Community and Junior Colleges (AACJC). the American 
Vocational Association (AVA), and major conferences of State vocational education associations. 
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Appendix B 



Analysis of Case Studies 



Case studies of exemplary business, industry, and labor involvement in the evaluation and 
planning of vocational education were conducted at three sites. These case studies provided a 
comprehensive base of information for developing guidelines that will work in local settings. Data 
were collected through the examination of documents, by open-ended discussions with appro- 
priate personnel, and by observations. Telephone discussions were conducted when necessary to 
vetify or clarify information. Additionally, specific strategies and procedures were identified for 
development, refinement, and dissemination. 

Methodology proposed by Patton (1980) and Guba and Lincoln (1982) provided the basic 
steps in designing the case study. These included sistting boundaries, defining the unit of analysis, 
selecting sites, establishing initial contacts, developing data collection systems, defining fieldwork 
procedures, collecting data, and analyzing data. 

The data from the case studies were collected and stored according to the respondent's role 
as practitioner, decision maker, and/or policymaker. These interviewees included members of 
chambers of commerce, manufacturers' associations, labor unions, personnel directors, advisory 
committees, department heads, governing boards, chief administrators, teachers, program direc- 
tors, and others considered to be participants in the development of human resources for the 
needs of industry. Following the completion of the on-site fieldwork, the information was analyzed 
for major themes or issues and for information supporting these themes or issues. A brief sum- 
mary of the information collected via the case studies is presented in appendix C. 

The two complementary approaches just described above provided the basis for the synthesis 
and integration of information. This synthesis and interpretation determined areas of agreement/ 
disagreement, identified gaps in the information base requiring additional study, and provided a 
basis for developing workable strategies, procedures, and techniques for inclusion in the 
handbook. 
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Appendix C 



Case Site Reports 



This section contains three case site summaries of exemplary efforts of public secondary and 
postsecondary schools to involve business, industry, and labor in vocational education planning 
and evaluation. Each site represents a special industrial setting, geographically and economically 
distinct, with its own unique educational structure. One site represents an eastern State in a labor 
market dominated by the manufacturing industries. Another site located in the midwest is charac- 
terized as having a diverse industrial base, and the third site in a rural western State is described as 
an agricultural area. 

The summaries were developed around the four major study questions: 

• What changes are needed in the planning and evaluating of secondary and postsecondary 
vocational education if input from business, industry, and labor is to be included? 

• What specific decision-making activities should vocational educators and representatives 
of business, industry, and labor undertake in planning and evaluating secondary and 
postsecondary vocational education? 

• How can vocational education, business, industry, and labor implement planning and 
evaluating activities in secondary and postsecondary vocational education? 

• What process activities address the involvement of vocational educators, and representa- 
tives of business, industry, and labor in evaluating secondary and postsecondary voca- 
tional education? 

The following conclusions were reached after the case site studies had been completed. 



What chinget ire needed In the planning and evaluation of secondary and postsecondary voca- 
tional education If Input from business, Industry, and labor is to be Included? 

Education staff directly responsible for a business, industry, and labor/education program 
should have (1) business, industry, labor experiences, and (2) experiences at various levels of the 
educational system (teacher, supervisor, principal, superintendent). Top management support is 
important and must be constant over time. This support includes a statement of philosophy and a 
mission statement that explicitly supports business, industry, and labor in educational involvement 
in planning and evaluation; attendance at meetings (business, industry, labor, education, commu- 
nity service organizations): and participation on business, industry, and labor boards. 

Schools serving the same labor market area must coordinate their efforts to pool resources, 
eliminate duplicated programs, and develop a unified approach to serving employers. Such coor- 
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dination can provide swifter response to industry requests and eliminate red tape. Schools also 
must be flexible with regard to scheduling and instructional style. Involvement in economic devel- 
opment activities tends to bring educators and business, industry, and labor closer together. 



What tptclflc d«cltlon-making activltl*! should vocational educators and representatives of busi- 
ness, industry, and labor undertake In planning and evaluating secondary and postsecondary 
vocational education? 

Craft committees or program steering committees can play strong roles in planning and evalu- 
ation. Every time they review a curriculum guide or evaluate a request for new equipment, they are 
involved in specific decision making, Educational personnel must often work with business, indus- 
try, and labor to help identify particular training problems. This includes identification of program 
needs, development of goals and objectives, specification of outcomes, participation in program or 
Instructional decision making, and shared input into financial program decisions. For ah activities, 
defining the rules and guidelines relating to individual group responsibilities is essential. 



How can vocational education, business. Industry, and labor Implement planning and evaluating 
activities In secondary and postsecondary vocational education? 

Involving business, industry, and labor at the initial stages of vocational education program 
development is basic to effective cooperation in planning and evaluation, Advisory committees or 
councils are one means of introducing business, industry, and labor input into planning and evalu- 
ation. The representatives of industry must recognize the constraints under which educational 
institutions operate. A variety of additional formal and informal contacts also are necessary to 
develop a spirit of mutual trust between education and business, industry, and labor representa- 
tives. For example, weekly breakfast meetings can contribute to such a spirit. Educational repre- 
sentatives who can commit their institutions to riespond to private sector suggestions and requests 
also facilitate viable input. 



What process activities address the Involvement of vocational educators, and representatives of 
vocational education, business. Industry, and labor In planning and evaluating secondary and 
postsecondary vocational education? 

One common role is responding to follow-up study questions about graduates. A periodic eval- 
uation audit of all programs, perhaps once every 5 years, involves business, industry, and labor in 
sufficient ways. Industry representatives can provide information about program completers, 
review the curriculum and equipment being used in the program, and compare the skills and Inter- 
ests of current students to the present and future needs of business, industry, and labor. Evalu- 
ation of customized training is another useful input. Generally, an oral or written summary of 
recommendations is prep ired for the purpose of improving the program. 



Case Site Summaries 



Case Site 1 

A summary of the key structural elements of the cooperation between education and business, 
industry, and labor in this area includes the following; 
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• An administrative component of the community colleges that focuses on outreach. 

• A "career education cooperative" that brings together all the vocational education 
providers. 

• A functioning private industry council that existed prior to legislative mandate. 

• One community college has weeldy breakfast meetings where business leaders learn 
about the college and what it can do for them. 

• Representatives from the education sector join chamber of commerce personnel in meet- 
ings with companies relocating to the area. 

• A long-range planning document-"Planning for the 80s"~was prepared that involved 
input from all sectors. 

• One company has loaned a half-million dollars worth of up-to-date equipment to the 
schools, and will continue to put in state-of-the-art equipment as it is available. This 
modern equipment will be used both for students in the vocational program and for com- 
pany employees' training. 

• Articulation between some secondary and postsecondary programs has taken place. 

• Schools share expensive equipment and facilities so that students from more than one 
school have access, while duplication is avoided. 

The basic ingredients (from the educator's perspective) to the successful cooperative effon 
appear to be the following: 

• Institutional flexibility, adaptability, and quick turnaround time 

• Emphasis on research, evaluation, and strategic planning 

• Constant personal contact with the business community 

• Active seeking of grants for customized training 

• Writing and dissemination of results 

• Trust between school staff members 

• Involvement in local economic development efforts 

• A marketing approach that stresses needs assessment, input, and evaluation 

• Active advisory committees 

• Use of Qvaluatlpn as both an auditing and feedback process 
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Cast Site 2 

The key structural elements of the cooperation between education and business-industry in 
this area appeared to be the following: 

• A strong cooperative education program 

• Required teacher contact with business 

• A strong advisory committee structure 

• Regular evaluation processes 

Some examples of cooperation among the parties that were cited during the interviews follow: 

*i '^ogree program (initiated at the request of business-industry) was developed and 
approved within a 3-month time span. 

• A motorcycle shop has vocational students place their orders for parts for use in school 
classes through them. The students have to drive across town to pick up the parts; they 
learn quickly to be careful in ordering. 

• Business-industry representatives help determine criteria for employment qualifications 
for teachers in new subject areas. 

• A training program for migrants was established on the basis of input from business- 
industry and funds from private foundations. 

• Business people are involved in vocational education curriculum development. 

• . The school takes its equipment to work sites for training of employees. 

The basic ingredients (from the educator's perspective) at this site that contribute to the suc- 
cessful cooperative effort appear to be the following: 

• An extensive staff development program for faculty requi ements for teachers to have 
contact with business-industry 

• Trust in faculty members to make changes and contacts on their own initiative 

• Individual autonomy of faculty members 

• An extensive co-op program 

• Making genuine use of advisory committee members, whether through regular meetings 
or individual contact by teachers 

• Active outreach efforts to business-industry 

• Surveys of employer needs by formal and informal methods 
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• Active seeking of grants for customized job training 

• Flexibility and quick responsiveness to requests from business 



Cue Site 3 

:<\ 

The summary of key structural elements of the cooperation among education, business, indus- 
try, and labor focused on a building trades apprenticeship training program that is a joint effort of 
business, industry, labor, and education. The school corporation operates the entity but industry 
funds it to a large extent. All groups are involved in its curriculum planning, student evaluation, 
establishing class size, and other policy decisions. The program has a long history of development 
and operation. Many of its administrators and instructors formerly practiced a trade. The advisory 
committee structure is strong and communication between all parties is frequent and open, even if 
there are concurrent difficulties in labor-management negotiations. 

Key elements supporting involvement in planning and evaluation include the following: 

• Strong leadership and commitment 

• Ongoing communication 

• An honest attempt to see the other side's perspective 

• A sense of ownership among all parties 

For example, the superintendent of schools is at every graduation event. Owners of individual 
companies will volunteer needed materials and supplies on a continuing basis to support the 
program. 

Evaluation and planning involvement encompasses both formal and informal activities. These 
include the following: 

• Evaluation primarily upon repeat requests from an employer for additional students or 
apprentices 

• One- and five-year follow-ups of students and apprentices 

• Planning input from advisory committee meetings and on a one-to-one basis between 
instructors, employers, and union representatives 
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